🎉 One Paper of Our Team Has Been Accepted by TMM

One paper of our team has been accepted by IEEE Transactions on Multimedia recently.

Research Background

In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. Systems in real-world environments constantly encounter adverse weather conditions that are not previously observed.

Therefore, it practically requires adverse weather removal models to continually learn from incrementally collected data reflecting various degeneration types.

Existing adverse weather removal approaches, for either single or multiple adverse weathers, are mainly designed for a static learning paradigm, which assumes that the data of all types of degenerations to handle can be finely collected at one time before a single-phase learning process. They thus cannot directly handle the incremental learning requirements.

Key Contributions

To address this issue, we made the earliest effort to investigate the continual all-in-one adverse weather removal task, in a setting closer to real-world applications.

1. Novel Continual Learning Framework

We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. Equipped with a principal component projection and an effective knowledge distillation mechanism, the proposed KR techniques are tailored for the all-in-one weather removal task.

2. Unified Network Structure

It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated in the unified network structure.

Experimental Results

Extensive experimental results demonstrate the effectiveness of the proposed method to deal with this challenging task, which performs competitively to existing dedicated or joint training image restoration methods.

The results show that our continual learning approach can effectively handle incremental weather conditions while maintaining performance on previously learned tasks, addressing the critical challenge of catastrophic forgetting in image restoration.

Conclusion

This work represents a significant step forward in practical adverse weather removal systems. By enabling continual learning capabilities, our approach brings weather removal models closer to real-world deployment scenarios where new weather conditions are encountered over time.

Congratulations to De Cheng and all co-authors for this outstanding achievement in IEEE Transactions on Multimedia! 🎊